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ABSTRACT 

The  change  of  initial  condition  (CIC)  problem  in  the  theory  of  smoothing 
in  linear  estimation  is  formulated  in  terms  of  a  fixed  rank  perturbation  to  a 
covariance  kernel.  The  CIC  or  partitioning  formulae  are  then  shown  to  apply 
for  general  nonstationary  processes.  It  becomes  clear  that  the  formulae 
involved  derive  from  inversion  formulae  for  fixed  rank  modification  of  a 
positive  definite  kernel  or  matrix. 
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SIGNIFICANCE  AND  EXPLANATION 


The  estimation  of  one  process  from  measurements  on  another  related 
process  is  a  problem  that  arises  in  many  areas  such  as  Time  Series  Analysis, 
Econometrics,  Communications  Engineering  and  Control  Engineering.  Particularly 
in  the  Engineering  Applications  there  is  a  great  interest  in  various  computa¬ 
tional  forms  of  the  algorithms  proposed  to  solve  the  above  problem.  One 
important  question  concerns  the  effect  of  perturbations,  in  the  initial  condi¬ 
tions  of  the  process  statistics,  on  the  algorithms.  This  article  discusses 
such  effects  for  general  process  models  and  relates  the  results  to  the  inver¬ 
sion  of  perturbed  matrices  or  kernels. 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MFC,  and  not  with  the  author  of  this  report. 


SMOOTHING  ESTIMATION  OF  STOCHASTIC  PROCESSES  -  PART  I: 
CHANGE  OF  INITIAL  CONDITION  FORMULAE 


V.  Solo 


Introduction ■  Recently  a  great  amount  of  attention  has  been  focussed  on  various 
algorithms  for  solving  the  smoothing  problem  of  linear  estimation  theory*  see  Ljung 
and  Kailath  [11],  Lainiotis  [10],  and  references  in  these.  The  main  algorithms 
include  the  two-filter  formulae,  Mayne  [13],  Fraser  [3],  Mehra  [14];  the  innovations 
formulae,  Kailath  and  Frost  [7];  and  the  partitioned  algorithms,  Govindaraj  and 
Lainiotis  [4] . 

This  work  is  the  first  part  of  a  two  part  investigation  of  these  algorithms. 
In  Part  I  it  is  shown  how  change  of  initial  condition  (CIC)  or  partitioning  formulae 
hold  in  a  very  general  setting  (the  CIC  problem  is  shown  to  involve  fixed  rank 
perturbation  in  matrix  inversion).  In  Part  II  the  nature  of  the  two-filter  algorithms 
is  explored  by  providing  a  simple  derivation  that  shows  to  what  extent  the  formulae 
hold  generally  and  so  reveals  exactly  how  a  wide  sense  Markovian  assumption  in  neces¬ 
sary  for  their  full  utility. 

The  remainder  of  the  paper  is  structured  as  follows.  Section  I  contains  a 
discussion  of  CIC  tormulae  for  discrete  observations.  Section  II  concerns  CIC 
formulae  for  continuous  observations  (actually  the  formulae  are  the  same) .  Section  III 
discusses  the  relation  with  other  work. 

Before  continuing  recall  the  matrix  inversion  lemma  (MIL).  Let  7,  ;  ,  D 
bo  matrices  related  by 


Z  =  +  I'D:" 


then 
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The  following  consequences  are  well  known 
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I .  Change  of  Initial  Condition  Formulae. 

Consider  the  linear  estimation  of  an  n^  dimensional  process  x(t)  from 
measurements  on  a  related  n  dimensional  process  ^(t).  We  deal  firstly  with  dis¬ 
crete  measurements  (at  times  t,  <  t_  <  . . .  <  t  =  T  in  an  observation  interval 

12  N 

?  t  T)  which  we  collect  into  a  vector  tj .  Suppose  the  process  x(t)  is  composed 
of  a  zero  mean  process  x^(t)  plus  a  transient  component  of  the  form  ;(t,0):'  where 
Mt,0)  (henceforth  denoted  £  )  is  a  deterministic  transition  matrix  (with 
viz  :  ;(0,0)  =  I)  and  t,  is  a  random  vector  uncorrelated  with  x^(t).  It:  sup¬ 

posed  that  the  transient  has  induced  a  perturbation  within  the  measured  process  y(t) 
so  that  v(t)  is  of  the  form  ^(t)  =  y^(t)  +  g(t,0K,  where  y  (t)  is  a  zero  mean 
process  and  £(t,0)  a  deterministic  function.  Thus  the  measurement  vector  has  the 
form 

=  tt  +  G: 

-  -a 


The  setting  described  here  has  been  used  by  Aasnaes  and  Kailath  [1]  to  study  the 
robustness  of  filtering  algorithms  to  init ial-condition  perturbations. 

Ia.  A  basic  CIC  Formula. 


Consider  now  the  linear  estimation  of  x(t)  from  the  measurements  tj  under 
varying  assumptions  for  the  statistics  of  the  £  vector  i.e.  the  initial  conditions 
for  x(t).  Vndor  condition  "0"  let  F^(r)  be  the  distribution  function  of  the  vector 

and  suppose 


.<:>  = 


-  m  )  {•  -  m  )  = 

-0  -  -0 


wiu  r"  F.  denote,  integration  with  respect  to  the  distribution  F^(:).  We  have  for 
<  xample 

=  it5o 

A  •  •  :t  ■ !  i  n- 1 1  y  w  ran  o-n;ute  the  linear  least  squares  estimate 


x„(:  •')  =  t’  (x  tt> )  •  (x,  ft) ."* ) 

0  -  <>  -0  -0  u  1 


*„ (♦ )  -  x(t)  -  !.(.(x(t )  )  -  x_(t)  *  (-  ■  :'n> 


"  -  .  CO 


;C  -  •-,) 


1  can  :it  ion::  .if  chant:...  "i  tc  "!  " 

:V-’  -  -1  '  V-  -  •-!’<-  -  -•/  •• 

ir.  cor.:  ntina  X,  (-  7)  iron  the  con-Ution 
the  formula 


Xj(t  T)  =  E  (x(t))  -  (x,  (tW'pr 


x,  ft)  =  x  (t)  -  I-:,  (x  (t ) )  =  x  (t)  *  •  (■  -  n,  ) 
-1  -  i  -  -a  -t  -  -1 


-i  -  -  hx(«)  -  ^  =  i'a  *  _•<_  - v 


CL)  that 

: :  j  ( x  ( t )  )  -  L.ixttn  ,  it  (r.x  -  2C) 

-I  *1  -ll  -1  '  -1  i -1-1 ; 

T:.1:.-  from  (la) 


5,  <*  T)  -  =j<5‘*a)  -  c,£l  ^ 


2,,(t  7)  -  ;V2(t))  -  7t§o  L'0 


)  ,  CO 


’  4'ii(t)  *  -t(-  - ”  -i' 1 


(X  (t  O'1)  •  ■  :• 

-a  -a  -t-1- 


(la) 


(ID 


(lc) 


■al culat i c 


Cn) 

(2b) 

Cc) 


O) 


(4) 


(5a) 

(5L) 


(Ca) 


-4- 


No  subscript  is  given  to  this  last  expectation  since  the  same  result  is  obtained  under 


V  (t.ojT)  =  P^t.olT)^1  =  ;fc  - 

Note  that  in  formulae  (8)  the  suffices  0,1  are  dummy  suffices  and  may  be  interchanged. 
Since  no  relation  has  been  specified  between  t  and  T,  equations  (8)  are  both 
filtering  and  smoothing  formulae.  It  is  interesting  that  the  data  dependent  portion  of 
the  correction  tern  in  (8)  is  the  same  for  all  t. 

Now  let  us  turn  to  find  a  CIC  formula  for  P^(t,o|T)  or  r^(t,o|T).  Since 
'  (O'T)  is  of  intermediate  interest  a  CIC  for  this  will  be  found  too. 


Ib.  A  CIC  formula  for  P1(t,0|T)II1 
Consider  then 


P^t.olT)^1  =  :t  -  M^G 


-1 


=  :t  ~  MoS^S  •  bv  <6c> 


it'-1  '  -  Sol^S 


=  :t(I  "  “2 ' by  (6d)  and  (MILd) 


=  'It  -  -o-o 's1 {l  '  "2,S“1S> 


=  Pgtt.olTl^hl  -  IG'S^G) 


let  us  call 


= .  -  "S's^sj =  ”  -  :-s*sr1sJi 


(9a) 


(10a) 


Eo(t'°'T,-:o  =  -o(t'0!:T> 


=  r  (t,o  =  f  (t.olTjir-1 


(9b) 


>n  the  other  hand,  guided  by  (MILe)  we  find  via  (6d)  that 


-1- 


r 


\:  ;  1  i-vit;  (oh)  allow?.-  this  to  he  written 


r  (t,t  T)  -  rr(t,t  T)  4  ir](t,0  T)^;(t,o-T) 


I  .:.ali\  from  n't hogonal ity 


ll  *  ^(0,0;  T)  a  jl  -  P  (0f0,T) 


■  -  ;A-1  b---  (10'-') 


si1  ■  --:1  *  a1  -  ^i)_1 


A  ■  A  +  <  -  V1  »  Lo  '  V0'0'71 


(10  5)  implies*  by  subtraction 


-1  ..-1  -1  .,-1 
P  -  =  p  - 

-1  -1  -0  -o 


The  Cull  set  of  change  of  initial  condition  formulae  are 


Equations 

(8a) 

.  (8b) 

for 

Xj (t ' T) 

Equations 

(9b) 

.  (9c) 

for 

(t ,0 | T) 

i 

Equations 

(U'a)  , 

,  (12b) 

for 

p  (t  ,t ;  T> 

i 

Equation 

(lCc) 

for 

- 

\  ■ 

Equation 

U0O 

f  or 

thut  *•'?*.  ry  quantity  in  these  equations  .has  a  stochastic  interpretat i on .  This 
/ r  a  rapid  comparison  with  other  work:  see  Section  III.  Also  it  is  shown  in 
T I  that  the so  equations  hold  also  for  continuous  time  measurement. 


If.  Calculation  of  v(t,o‘T)  in  continuous  measurement  State  Space  Models. 

The  quantity  *  ( t  #  0 1 T)  =  P(t,0;T)H  ^  has  only  appeared  in  earlier  work  with 
T  =  t  or  t  =  0.  Here  we  derive  a  differential  equation  for  ■?  (t,0;T).  Begin  with 


the  usual  State  Space  Model 


dx/dt  =  F (t) x (t )  +  T (t)w(t) 


y  (t)  =  H  (t )  x  (t )  +  v  ( t ) 


where  w  v  are  uncorrelated  white  noises  with  covariance  matrix  I.  We  use  a  formula 
of  Rauch  et.  al . ,  [21 J,  (also  Kailath  and  Frost  [7,  equation  (34a)] 

dx(t;T)/dt  =  F(t)x(t[T)  +  r(t)r’ (t)p-1(t)  (x(t;T)  -  x(tjt)) 


d :  (t  ,o[  T)  /dt  =  d/dt  Efx(t)  -  x  (t  J  T) )  x'  (0)  .f'1 


=  Fft)E(xft)  -  x  ft  ] T) ) x *  (0) 


-  r  (t)  r  •  (t)  P  (t)  [r  (t  ,0.  t)  -  v  (t  ,0  i  T) ) 


di  ft ,o| t) /dt  =  (f (t )  +  r (t ) r •  (t)p_i(t))t  (t,o'T) 


=  r  (t)  r  •  (t)p  (t)f_(t,oit) 

We  need  only  augment  this  with  an  equation  for  i(t,o|t)  which  is  easily  shown  to  be 
the  Kalman  Filter  transition  matrix 

d_(t,0|  t)  /dt  =  (F(t)  -  P(t)H' (t)H(t) )  :  (t,0' t) 


with  initial  condition  t(0,o[0)  =  I. 


r  f°™ula  for  a  Causi-ian  likelihood . 

ronsidr-r  th.e  log  Caus-^ian  likelihood  which  is#  to  within  a  constant 

in  ^  »  in  |§1!  -  i 
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Now 


|s1l  -  !s0  -  S'AGl  -  isQllx  *  s^V-- 


=  |s0!  |X  +  -IS's^sl 


IsJlAllA'1  +  0Q\ 


=  |g0l|A||f  I 


by  (10c) 


'life'll  =  %  ~  S<!?i  "  2o)r-i  L'i 

=  'fo^L'i  ‘  (-i  '  V-i 


where  :i  -  ijWlT)  »  see  (7a). 


L/'s’1^  -  iAi'fo1  -  -l1---’ -Q1' ^  ■  V-i 


by  (Ml Lb) .  Continuing 


-  aiSo^o  *  -i--o  '  ('1  '  -o’’-i 
=  i r0i0l‘i0  ~  (-i  ‘  V-o '  -i~o '  l-i  ‘  -0’  -1 


since  L\  -  L'0  ‘  £(2l  "  V  ' 


Now  denote 


-  v  ♦  v*  -  v  ♦  *' 

+  (“i  -  20>  1  (-i  ’  -o’ 

x1  =  x^  (0 1 T)  ;  x0  =  x0U-|T) 

p1  =  Pjto.ojT)  ;  r0  =  'V°'°]T' 


ar.h  set  t  =  0  in  (8a)  to  see 


Continuous  Time . 


II. 


Let  us  retain  the  assumptions  of  Section  I  concerning  the  presence  of  the  transient 
term  in  both  x  and  y  and  suppose  now  that  y  is  measured  continuously  over  the 
interval  0  t  <_  T.  Introduce  similarly  to  before  the  kernels  m  ,  S^,  ghg',  c/.g* 
defined  on  [0,T]  [0,T]  by 


where 


M.  (t,c)  =  E.  (X.  (t)y.  (0)  ) 

1  l  'i 

i  =  0,  1 

(15a) 

=  E.(y.(t)v!(:]] 

i  =  0,  1 

(15b) 

gg'(t,,-)  = 

(t,0)  =  it_g' (J) 

(15c) 

x.  (t)  -  x(t)  -  E .  (x  (t ) )  i  =  0,  1 

-l  -  i  - 

y. (t)  =  y(t)  -  E.(y(t>)  i  =  0,  1 


Then  it  follows  as  in  Section  la  that 

sx  =  s0  +  g.ig'  (I5d) 

Mx  s  Mq  +  -'9 1  (15e) 

suppose  too,  without  loss  of  generality  that 

~  is  positive  definite  (15f) 

••►-•call  ci(t)  is  an  n  *•  n  matrix  function.  In  the  sequel  h(t),  f(t)  will  denote 
*  V  x  " 

general  n_ -vector  functions. 

The  appropriate  setting  for  the  further  development  of  CIC  formulae  in  a  general 
fashion  is  the  theory  of  Reproducing  Kernel  Hilbert  Spaces  (RKHS) :  See  Parzon  [16] 
and  r’ailath  ["}.  The  idea  in  the  RKHS  approach  is  to  provide  a  coordinate  free 
representation  of  continuous  time  equivalents  of  discrete  forms  such  as  "h'S^f." 

::ow  Parzon  [16]  only  discusses  KKHS's  for  scalar  processes  so  it  will  be  indicated 
rirfly  how  to  define  an  appropriate  RKHS  for  vector  processes. 


-14- 
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Proposition  1.  Existence  and  Uniqueness  of  a  vector  RXHS  generated  by  an  n  *  n 
matrix  covariance  kernel  (cf.  Parzen  [17,  Theorem  3.1]).  Let  y(t),  t  in  [0,T] 

have  covariance  kernel  ;  (t,c).  Let  H(l)  consist  of  all  n  -vector  functions  h ( * ) 

-  y 

on  [0,T]  of  the  form 

h(t)  =  E (y (t) U) 

for  some  scalar  U  in  SL0(y(t),  t  in  [0,T])  where  SL^  is  the  (scalar)  Hilbert 
Space  of  random  variables  that  are  finite  scalar  linear  combinations  (of  the  form 
a^y(t  ))  of  random  variables  y(t),  t  in  [0,T]  or  limits  in  mean  square  of  such 
finite  linear  combinations.  Define  an  inner  product  on  H(T)  as  follows.  If 
h(t)  =  E(£(t)U);  f(t)  =  E(y(t)V)  for  U,  V  SL2 ,  define 

(  h*  ,f  >_  =  E(L’V) 

Then  HO  is  a  Hilbert  Space  satisfying 

(a)  I.lst)  ,  HO  i  =  1,  . ..,  ny 

where  "  (»,t)  is  the  ith  column  of  r  ( * ,  t ) 

(b)  (h'^lvt))  =  h  (t)  for  all  h  HO 

Proof .  Follows  much  as  in  the  proof  of  Theorem  3.1  of  Parzen  [17).  Observe  for 
example  though  that 

=  E(y<  -)y.  (t) ) 

where  y  (t)  is  the  ith  component  of  y(t).  Thus 

If,  .(•,!))  =  Ely.  (t)V)  =  f .  (t) 

-l  l  l 

remark  1.  Let  K(t)  lx’  an  n  •  m  matrix  function  on  the  form  F(t)  =  (f.  (t)  ...  f  ( t ) ) 

- -  y  -  —  1  m 

where-  f.(t)  arc  n  -vector  function?.  Also  lot  G(t)  bo  a  similarly  defined 
- 1 

n>t  •  n  matrix  function.  With  a  slight  abuse  of  notation  denote  by  the  m  >  n 

matrix  whe.-r  i,j  element  i r  f  f!.ci  .  Thus 

-i-i 

<  h * ,  .  («,t)  >  =  h  (t) 

_  *•  ] 
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can  be  written 


Also 


reads 


Finally  we  denote 


by 


<  h' >£  =  h*(t) 

<  t),s.(.|S)>  =  .  (t,S) 

-l  -j  I  l] 

<  n*,t)  ,z(-,s)  >  =  r  (t,S) 


fj,  (  H(Z)  i  =  1,  . . . ,  m 

F  e  H(Z) 


Remark  2 ■  All  the  scalar  RKHS  theorems  given  by  Parzen  [16]  apply  to  vector  RKHS's. 


The  basic  question  in  applying  RKHS  theory  consists  in  finding  computational 
expressions  for  the  RKHS  norm:  for  some  concrete  examples  see  Parzen  [16]  and  Kailath 
et.  al.  [7],  To  obtain  results  analagous  to  those  of  Section  I  we  need  a  MIL  for  a 
RKHS.  First  however  it  is  convenient  to  define  the  direct  product  of  two  vector  RKHS's 
(cf.  Parzen  [I7J;  also  Kailath  (6,  p.  18] ). 

If  2,  T  are  two  m-dimensional  positive  definite  kernels  on  [0,T]  *  [0,T] 

generating  RKHS's  H(Z),  H(Z)  respectively  then  the  direct  product  H(Z)  ®  H(.)  is 

the  space  of  m  '  m  matrix  functions  on  [ 0 , T]  x  [0,T]  which  are  linear  combinations 

of  products  of  functions,  one  from  HIZ) ,  the  other  from  HIT)  (of  the  form 

n(t,s)  =  (  .  :  .z  .  (t)  -'  (s)d.  )  on  limits  in  the  norm 
i  l-i  -1  U 


,m( .  ,x)  |;‘ 


Z-  Z  ■<  z!  ,z  .  >  <  3!  ,0  ■  >  d 

1  1  -1  -1  z  -1  -3  :  13 


of  such  linear  combinations  (d  are  real  scalars). 

U 

Proposition  2.  MIL  for  RKHS's.  Let  Z,  be  m  x  m  matrix  kernels  on  [ 0 , T]  '  [0,T] 
related  by 


2(t,s)  =  ;  ( t , s )  +  y (t ) nr ' (s) 
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where  ]_(t)  is  an  m  ■  d  matrix  function  and  :  a  d-d  positive  ie finite  matrix 
Let  H(Z),  !i  t  bo  the  '-JtHS's  generated  by  Z,  respectively  then  iff  (  ) 

(ie.  jc  H  ( .. )  ,  i  =  i,  d  whore  i,(t)  ir  the  it:,  column  of  lift)  we  have 

(a)  f  •  H  ( Z)  iff  f  •  HO 

This  does  not  mean  H(Z)  is  a  subs;  ace  of  H(  1  or  vie,  versa  sine.,  that  would 
entail  equality  of  inner  t  redact -■ . 

(b)  For  h,  f  11(2)  the  H(Z)  inner  ;  r  -duct  is 

<  h'  ,f  „  -■  (  ,  y'h  •  (f  ' 

where  V  =  D  ^  t 

Remark  1.  Formula  (b)  was  suggested  by  (XI La) . 

Remark  2 .  Note  that  V  is  well  defined  iff  ;  HO)  i.e.  O',  • 

Proof .  To  see  (a)  we  use  a  theorem  of  Aronsz-  ,  .  354]  which  implies  that  (a) 

if 

(i)  There  are  constants  c,,,  c,  such  that  c^Z  -  :  and  c.h  -  Z  are  jositive 
semidef in ite . 

In  fact  (i)  is  true  witli  =  1  and 

c.  =  tr  U  V’ (10) 

where  •  =  largest  eigenvalue  of  0.  This  claim  is  p rover,  in  At  pend  ix  B. 

Now  in  view  of  (a),  (b)  is  established  if  it  is  shown  that 

(ii)  Z(.,t)  HC)  for  all  t  in  [0,7] 

(iii)  for  all  vectors  h  H(  ) 

(  V ,Z(  .,t)  >  =  h* (t) 

Now  (ii)  is  clearly  true  since  .  (*,t)  •  H(),  H(  )  and  Z(.,t)  =  ( • ,  t )  * 

.(•in;1  (t )  . 

To  see  (iii)  let  h  H(  )  and  compute 
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,Z(*  ,t)  >  =<h',2(-,t))„  -  <  h'  1‘ ,  Z(.  ,t)  >. 

=  <  h*  -  <  s'  .roo  (t) 

-  <  v  -  <  h* .ro'1*  ori-JO'  <fc> >- 

=  h'(t)  +<h’,r>„[i  -  v"1n*1  -  v'"<  (t) 


=  h '  ( f  ) 


Since-,  by  definition  of  V 

-1  -1 

v  ( +  ( ?  * ,  r > . )  -  i 


The  result  is  thus  proven. 
Remark  3.  Since  7  H (7)  and 


r (t)pr' (s) 


cd  . :d 

1=:  3=1  13*1 


(t)  ■ 


(s) 


it  follows  that  rD£’  •  HC.)  8  HO  i.e.  (iv)  holds 
(iv)  Z  -  :  HO  8  HO 

how  however  according  to  a  result  of  Godaira  [15]  (see  also  Kailat'n  [6,  p.  20])  (i)  , 

(iv)  are  necessary  ar.d  sufficient  for  the  problem  of  "detection"  between  Z  and  Z 

to  be  nonsingular.  It  is  apparent  that  7  •  HO  ensures  this  nonsingularity. 

Hemark  4 ■  In  a  similar  way  other  MIL  expressions  analogous  to  those  of  Section  I 

(viz:  MIL(fc)  -  MILO)  can  be  obtained  (iff  7  HC)). 

Mow  these  results  can  be  applied  to  the  present  problem. 

Proposition  3.  Let  H.  be  the  FKHS'3  generated  by  S.(t,.:),  i  =  0,  1.  Suppose  that 
__r - i  -t 

is  positive  definite.  Then  iff  g  H  we  have 
(a)  h  H„  iff  h  ■  Hj 
(o)  The  H  inner  product  is  given  by 

=  (h’,f>0  -  (h'»2>02  1(2'>£>o  (17a) 

r.-  :  =  i"1  +  'g'.g>0 
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Proof.  Since  1  is  assumed  positive  definite  the  result  follows  from  Proposition  2. 
Remark.  For  future  use  we  state  the  result  corresponding  to  (MILb) .  Under  the  condi¬ 
tions  of  Proposition  3 

<  h* ,ff>  «  <  h* ,f>  -  <  h' ,g>1A<g' ,f >Q  (17b) 

Finally  we  turn  to  consider  the  linear  estimation  problem.  The  RKHS  framework 
allows  a  general  solution  to  this  problem:  namely  (Parzen  [20]) 

x^t/T)  =  E  (x(t))  +  Ui  (t  j  T)  i  =  0,  1 

where  U. (t  T)  is  the  unique  solution  to  the  eauation 

-i 

E.  (U  (t  T)y!  <-))=:*..  (t,:)  =  E  (x.  (t)V  ('))  for  all  c  in  [0,T]  i-0,  1  (18a) 

1-1  -l  -l  l  -l  -i 

where  v  ^  ( ? )  =  v(:)  -  9  (- ) ^ 1  i  -  0,  1 

That  is,  U  . (t  T)  (the  jth  element  of  U.(t'T)  is  the  unique  member  of 

-12 

that  satisfies  (18a)  .  V.’e  can  write  symbolically 

■_x(t  T)  •:  y.  (tf.)  ,£  >  i  =  0,  1  (18b) 

The  equality  is  symbolic  because  (Indeed  h  ^as  variance-- 

this  is  clear  since  the  variance  in  the  discrete  case  is  Nn  ) .  In  any  case  we  are 

y 

now  ready  to  establish  the  CIC  formulae  analogous  to  those  of  Section  I.  To  see  the 
tyre  of  argument  needed  the  following  result  is  established  (cf.  the  main  result 
(6e),  da)  of  Section  la). 

Proposition  4.  For  the  situation  referred  to  at  the  beginning  of  this  Section  and 
under  the  conditions  of  Proposition  3  (in  particular  that  g  •  H^)  we  have 

Xjtt'T)  =  x0(t;T)  +  (:(t)  -  <M1(t,.),2>1)  (i>0(0|T)  +  rat  -  raQ) 
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Proof .  To  simplify  the  argument  and  help  reveal  the  basic  idea  we  take 


-1  "  -1  '  ' 

so  that  x1(t)  =  xQ(t)  =  x  (t)  and  (t )  =  yQ(t)  =  y(t)  and  x^lt'7)  =  7 .  (t  T) 

i  =  0,  1.  Let  us  denote 

=  b0(tiT>  +  (J(t>  -  <«i-2>1>4Y0 
-0  =  -o(°!T>  ' 

We  have  then  to  show 

'^(tjT)  =  (t  iT)  -  <K1,X>1 

since  L^(t|T)  is  unique  this  will  follow  if  we  establish 

(U^  (t  |T)^'  (c)  )  =  .^(t.a)  for  all  o  in  [0,7] 

Suppose  it  is  shown  that 

(i)  (U0(t  ;T)^'  (J) )  =  <  l!0<t .  * )  , ( • ,  -)  >0 

(ii)  E1(VQy,(o))  =  <3,»§1(-#3)>0 
Then  we  can  find 

E1(U1(tlT)J;1  (o))  »  <K0(t,*),S1(-,C)>  +  ($(t)  -  <  (t ,  •)  >„ 

=  (^(t  ,•),§!  (-,o)>0  +  <«u2',Si(-,-)>0  -  Cj3  (t,*),n>1i(g’,S1(-,c)  >c 
=  <M1{t,*),S1(‘,o)>0  -  <  Mx  (t .  ♦ )  .  3>1i<2,<  S1(-,o)>0  by  (15e) 

=  <M1(t,-),si(-,J)>1  by  (17b) 

=  M^(t,a)  by  the  reproducing  property 
Thus  we  have  to  prove  (i)  ,  (ii) . 

Let  P^ ,  PQ  be  the  probability  measures  induced  on  the  space  of  sample  functions 
of  the  normal  process  jr(t)  under  conditions  "1",  "0"  respectively.  Kow  according 
to  Remark  3  following  Proposition  2  the  assumption  g  ensures  the  problem  of 

detection  between  "1”  and  "0"  is  nonsingular  so  that  the  Radon -Nikodym  derivative 
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expression  (14)  of  Section  If  we  obtain  the  Radon -Nikodyro  derivative 


dFl/t5Po  =  ii!l~  *1  txP  U) 


where 


\  <0  ]  T)  -  x^tojT))  ■  (Px  (0 ,0  [T)  -  P  (0,0|T))  1(x1<o|T)  - 


I  (?i  -  -  V 


x0(o!t)) 
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Relation  with  other  work 


"ow  we  demonstrate  how  these  general  results  include  lainiotis'  generalized 
Partitioned  Algorithm  (GPA)  (see  Govindaraj  and  Lainiotis,  [4])  as  well  as  work  of 
Ijur.g  and  Kailath  [11]. 

Consider  the  state  space  models 


X  ( t ) 

=  f  (t)x(t)  +  r  (t)w(t) 

y  (t) 

=  H  (t )  x  (t )  +■  v  (t ) 

x.  , 

=  F  x,  w. 

-k4i 

-k-k  -K-k 

-k 

=  5k*k  +  -k 

where  m  each  case  w,  v  are  u:. correlated  white  noises. 

The.-,  each.  1-  of  the  form 

x  =  Xa  +  ~ 

y  =  va  -  gr  =  (Kx  +  v)  +  H:-; 

where  ,  y  arc  zero-nean  processes  and  f  =  x(C).  Also  in  each  case  ;  is  the 
transition  matrix  associated  with  F.  Thus  the  CIC  formula  derived  above  are 
applicable  to  these  state  space  models-. 

Ilia.  Relation  to  1TA. 

The  partitioning  view  of  the  TIC  problem,  begins  with  condition  "C"  and 
imagines  condition  "1"  as  produced  by  adding  a  further  transient  to  the  signal  and 

chservnt i~n  :  recesses .  Thus  in  conation  (5a)  introduce  m  -  ra,  -  m. .  Also  recall 

-r  -1  -0 

(did  and  denote  [_('  T)  =  _  to  find 

x.  (f  T)  =  x  (t  7)  *  ^(t,n  T)T  (O'T);'1  (m.  t  ^_n(C'T)) 

X  (t  T)  =  x.(t  T)  +  1  (t,0.T)x  (O'T)  (20a) 


Px  (t  ,t  |T)  =  P0(t,t|T)  +  V0(t,0|T)Pr(C|T)f  (t,0|T) 


(20b) 


where  we  have  introduced 

x  (0|T)  =  (0 ;t) (A0  CO |T)  +  (20c) 

Finally  recall  (10c)  in  the  new  notation 

P_1  (0|T)  =  0„  +  t'1  (20c) 

Now  recalling  the  stochastic  interpretation  of  all  the  terms  in  equations  (20)  set 
T  =  t  to  see  these  equations  can  be  compared  with  equations  (41)  ,  (42)  ,  (43)  ,  (44) 
of  Govindaraj  and  Lainiotis  [4]  (in  their  notation  take  i  5  0,  k  s  t;  "n"  =  "0" 
and  other  obvious  equivalences) .  Also  equations  (9c),  ( 1 Of ) ,  (11a)  can  be  compared  to 
equations  (55),  (56),  (57). 

The  case  =  0  is  discussed,  for  completeness,  in  Appendix  C. 

mb.  Relation  to  work  of  Ljung  and  Kailath  112]  . 

Equations  (9b),  (lOf)  can  be  compared  to  Ljung  and  Kailath  [10,  equations  32,  33) 
if  we  make  the  equivalence  W(t)  s  0  :  to  see  this  recall  the  stochastic  interpretation 
of  0  in  (lod).  (There  thus  appears  to  be  a  miswrite  of  equation  (33)).  Also 
equation  (8b)  can  be  found  in  Ljung  and  Kailath  [12]  in  two  cases:  for  T  =  t  see 
equation  (14) ;  for  t  =  0  see  equation  (26)  and  recall  that 

^(o,o|t)  =  (0,0  |t)  n"1 

=  I  -  by  (12c)  . 
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Conclusion .  In  this  article  change  of  initial  condition  formulae  have  been  presented 


for  linear  imates  of  general  nonstationary  processes  perturbed  by  fixed  rank 
transients.  The  formulae  apply  to  discrete  or  continuous  measurement  and  to  estima¬ 
tion  in  continuous  or  discrete  time.  Also  a  CIC  formula  for  a  Gaussian  likelihood 
has  been  given. 

The  derivations  depend  essentially  on  a  simple  Matrix  Inversion  Lemma. 


Appendix  A,  A  Cl C  formula  for  p^(t,t|T). 
First  observe 


P^t.tiT)  =  E:(*  Ct(T)  -  Ej^fxCtnHXj^ftjT)  -  E1(x 


(x  (t ) ) )  ’ 


- 


The  idea  is  to  establish  a  relation  with  Jl  S  However  let  ut  calculate 

P  (t,t!T)  =  E  (x(t)  -  xa(t(Tn  tx*  (t)  -  XjItjT)] 

=  E^(x(t)  -  Xj (t j T) ) x1 (t)  by  orthogonality 

=  ExIx(t)  -  E  (x(t)))  [x*  (t)]  -  E1(x1(tiT)  -  E1(x(t))]x'(t) 

=  E[x  (t)x'(t)]  +  j  "  -  P  (t,t]T)  by  orthogonality 

—a  -a  — t. — i“t  —1 


Thus 


Pjtt.tlT)  -  P  (t.tjT)  =  ~  IP_J  (t  ,t  jT)  -  P0(t,t|T)l 


Now  if  it  is  shown  that 

Px<t,tlT>  -  P0(t.tjT,  =  -  -0-0^-0 

= '  <-t  -  -  s’So1^’  + 

then  the  result  (12a)  will  follow  in  view  of  (7c). 

Consider  then 

V^i  “  :1(S?  ‘  S-10.1C,*S-1)MJ  by  (MILb) 

"iso  ::i  "  "i£i  G'l’’so  '‘i 

*  l;'o  -  V'G,)So1;ti  '  ■■hsilc:-G,sl1;’i  by  (6c) 

*  ^ 
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=  S  S^Mj  +  M^GAC'  +  ("'t  '  «1s'1G)AG’S^1M^ 

■  \S'o\  +  VolG^t  +  <*t  -  V^GIAG-S^GAC-  ♦  (vt  -  M1S-1G)-.G-S-1H' 


Sow  recall  (9a) 


*t  -  MjS-'G  =  (♦,  -  M0S^G)  (I  -  iC'S^G) 


which,  by  (10b)  is  also 


=  (?t  -  MqS-W  +  Ag-S^G)'1 


Thus  the  sum  of  the  second  and  third  terms  above  is 


Vo^'t  +  (5t  -  Volc)(I  +  •■c-S-lG)-1(^'S-1G)At. 


■  MosolcAct +  “t  -  vo1g,(i  - (i  +  ag,so1g,‘1)‘-;; 


■  moso1g4^  +  (ct  -  VolGK'-‘t  ‘  <‘t  ' 


v~*t  -  (:t  -  v?6>4tt 


hsl\  -  *osl\  +  Vft  -  (:t  -  'hsllc)!tl  +  (:t  -  ^i^O.IG’S-'m- 
*  \sl\  +  :tA?t  -  <‘t  -  V;1(;)A(:t  ' 


which  is  what  was  required  to  be  established. 


Appendix  B.  Derivation  of  (16) . 

Let  a.  i  =  1,  . ..,  N  be  any  m-vectors  and  t,  <  t_<  . . .  <  t  be  an 
—  1  1  7  M 

points;  in  [0,T]  .  The  claim  is  that 

E  =  IN  ^irit.lDf'lt.la  <  tN  a !  £  (t .  ,  t . )  a 

- -  ’  —  -  L  1=1  J  =  l-1-  1  2  -J 


i=l  i  —  j  -j  -  D 


Now 


E  =  aT(t.))D(rN  r*  (t . )  a  . ) 

1=1-1-  1  -  3=1-  3  -3 


u  ‘  Du  <  X  Qu ' u 


where 


u  =  I.  ,r’(t.)a. 

1=1-  l  -l 


Next  apply  the  reproducing  property  to  see 


“  =  £i=1<r* (•),!(•, t.) J^a. 


=  <r,(.).ii.1(.,ti)«1>l 


-  <£'(•) ,wN ( • )  >£  say 


Now  u  has  kth  element  u,  where 

k 


Thus 


However 


u^  =  is  the  kth  column  of  r 


a’H  =  :k=i\  i 


%'Vr  ‘  <Ji=i2i=‘-'ti>'^=iS<*'ti)2i>: 


=  ri-iE?-i2i<£,*«ti>'S‘-'tj>  >i-j 

=  ,1.”  .a!£(t.  ,t.)a. 

i=l  3=1-1-  i  3  -3 


by  the  reproducing  property.  The  result  is  thus  established. 
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Appendix  C . 


delation  with  partitioned  algorithm  for  !!  -  0. 

Consider  .'*w  the  case  that  !!  =■  C.  Then  by  (10a) 

2  (o|t)  =  -  ,i£7,  A 

r  -  — 1- 


=  E>  (0,0  IT}  = 


Thus,  since  =  l,  (9b)  becomes 

j.1(t,o|T)n1  =  i  Q  <t ,  c  ]  T)  r1 

while  (9d)  reads 

i_1tt  ,0|T)  =  io(t,0|T)  -  i0(t,0|T)^  ^ 

Also  (10c)  becomes 

p"1  =  r1  +  o 
-1  -1  1-0 

When  further  =  0,  (7b)  implies 

x0(°|T)  =  0 

Thus  for  t  =  0  (8a)  becomes 

X1  (fl  jT)  =  +  \  <0  jT)  1 


(Cl) 


CC2 ) 


(C3) 


(C4) 


The  pair  (C3)  ,  (C4)  with  T  =  t,  are  just  a  general  version  of  Lainiotis'  original 
partitioned  algorithm:  Lainiotis  [91,  equations  (3),  (4).  Equations  (Cl),  (C2)  car. 
be  compared  with  Lainiotis  [91  equations  (2h),  (21). 
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